<<<<<<< HEAD ======= >>>>>>> a4f724d0933cfc4b2322ddb3b951beb784e55322 <<<<<<< HEAD ======= >>>>>>> a4f724d0933cfc4b2322ddb3b951beb784e55322

Introduction

<<<<<<< HEAD

In this paper we investigate recent changes in the way archaeology is being done, especially the computational aspects.

=======

In this paper we investigate recent changes in the way computational archaeology is being done. Revolutions or paradigm-shifts in archaeology have been discussed to happen following theoretical discurses(???; ???), but there has been an argument in philosophy of science whether revolutions in science are either ideas-driven or tool-driven (???; ???). In this light we explore how many scientific disciplines have started to embrace methods to improve the reproducibility of their research and whether this change is part of a paradigm-shift. We focus on one set of tools for this change which is based on writing scripts in free and open source programming languages and review bibliometric evidence for this change in the sciences generally and in archaeology in particular. We evaluate how computational approaches to improving reproducibility and transparency in archaeology are mediated and transformed by digital approaches and test if the adoption of these new tools may be considered a revolution or some other process.

>>>>>>> a4f724d0933cfc4b2322ddb3b951beb784e55322

[need some more in here]

Revolutions in Science: Ideas-driven or Tool-driven?

<<<<<<< HEAD

There is an argument among philosophers of science that revolutions in science are either ideas-driven or tool-driven. One of the most widely-known models of ideas-driven change in science is Kuhn’s [-@Kuhn_1962] effort to describe the history of science by proposing paradigm shifts resulting from revolutionary change in communities of researchers. In brief, this model proposes that the history of science consists of long periods of tradition-bound ‘normal science’ punctuated by short episodes of revolution. Normal science was described by Kuhn as stagnant, routine day-to-day research focused on ‘puzzle solving’. What counts as suitable puzzles, and acceptable solutions to these puzzles, is governed by the norms and procedures of the prevailing paradigm. Revolutions in science occur as punctuations in the equilibria of normal science, when anomalous observations culminate in changes that replace one paradigm with another in a conversion or speciation process where the community adopts a new way of doing science. There has been substantial debate about how to draw a clear distinction between so-called normal and revolutionary science. [@watkins1970against; @toulmin1970does; @Casadevalle00158_16]. A notable criticism is that Kuhn’s normal science is unscientific, a situation where critical science had contracted into defensive metaphysics resulting from the domination of a ruling dogma [@popper1970]. This is at odds with a view of science that it is in a continuous state of revolution, with researchers simultaneously employing several styles of thinking and doing science, and continuously confronting theory with evidence and modifying their ideas based on the outcome. In this view, based on the post-World War II era of science, radical discontinuities are rare, and major developments instead emerge from the division and recombination of already mature fields [@andersen2013second]. Given the rarity of radical discontinuities, some biologists have argued that change in biology is better characterised by new ideas (e.g. Mendelian heredity, Darwinian evolution, and molecular genetics) replacing not a former paradigm but a conceptual vacuum [@wilkins1996are].

In archaeology we have seen claims for paradigm shifts in several areas. The most wide-ranging application is Clark’s [-@Clark_1993] comparison of North American archaeology to European archaeology during the 1970s-90s. Clark, similar to Binford and Sabloff [-@binford1982paradigms], identified the North American archaeological paradigm as rooted in anthropology, founded on post-positivist biases, and distinctive with it critical-realist ontology, a modified-objectivist epistemology, and an experimental-manipulative methodology. In contrast, the European archaeological paradigm is described a kind of history, mostly in a strict empiricist tradition, and typified by a realist ontology, an inductivist epistemology, and by an observational methodology. Clark’s analysis describes a multi-paradigm situation where multiple research communities (i.e. North American and European) simutaneously operate in different paradigms (which Kuhn noted as a sign of a immature discipline). His account is not a complete Kuhnian analysis because it lacks a revolutionary change event where one paradigm is replaced by another, although he hints that processual archaeology may be considered a radical discontinuity relative to the culture-historical approaches that preceeded it [@Clark_1993, pp. 206].

A more thorough treatment of the claim that processual archaeology was a paradigm shift is provided by @Meltzer_1979. Meltzer reviews literature arguing for a revolutionary change in archaeology during the 1960s and 1970s, and considers these claims in light of classic Kuhnian revolutionary events such as the replacement of the Ptolemaic system by the Copernican system, and Newtonian dynamics by the new Einsteinian dynamics. He finds that the changes that occured in Anglophone archaeology after the 1960s were incremental, and primarily of method, rather than a widespread replacement of one ontological structure by another one, incommensurable or incompatible with the former. Trigger [-@Trigger_2006, pp. 538] came to a similar conclusion at the end of his broad survey of the history of archaeology, that changes throughout archaeology have been mostly additive with only partial replacement.

We see less formal mentions of paradigm shifts in archaeology in reference to processual to post-processual archaeology [@Härke_2002], as well as more localised and specific shifts in archaeological practice and thinking. For example, @Snodgrass_2002 claims that among Classical Archaeologists, an increase in research on previously neglected periods odf antiquity, such as the Greek Early Iron Age, constitutes a paradigm shift. @McAnany_Rowe_2015 propose that the appearance of community-based participatory models of research among some communities of archaeologists is a paradigm shift, though they concede that it is more transformational rather than revolutionary. @Fuller_2010 has used the term to describe a shift in thinking about the emergence of agriculture as a parallel, simultaneous event at the start of the Holocene (the Neolithic Revolution), to new concept of the emergence as a protracted and entangled process, happening 20 or so times independently, following different pathways. Most of these paradigms are quite remote in meaning from Kuhn’s classical formulation, and this highlights an important complication with using Kuhn’s paradigm shift approach, both as a tool to understand the history of science and as a guide to changing it, which is Kuhn’s multiple definitions of a paradigm.

Masterman [-@masterman1970does] has documented 21 different uses by Kuhn, which she organises into three groups. First is the metaphysical notion of a set of beliefs, and this is the sense that dominates most of the critical writing on Kuhn’s model. Second is the sociological sense of scientific habits, the universially recognised scientific achievements that are the foundations of day-to-day normal science. These include specific attention-grabbing successes that provide a touchstone for the coordination of future research. Third is the concrete sense of an actual textbook, instrumentation or toolkit. This recognition of the multivalent nature of paradigms is important because it broadens the locus for scientific revolution to include sociological practices of science, such as might be produced by changes in norms of publication and peer review, and to include the physical infrastructure of doing science, such as the technology for collecting and analysing data, and for communication and collaboration among researchers.

We can see a compelling exploration of the concrete sense of a paradigm in the work of @Galison_1997, who has offered a tool-driven view of change in research communities. Galison analysed of the role of tools in twentieth century particle physics, starting with hand-crafted cloud chambers and bubble chambers and ending with digital counters of and computers. He identifies a profound change in physics when analog devices producing pictures were superseded by digital devices producing numerical data. At the heart of this change is a shift from an intuitive approach, stimulated by visual and pictorial model-building to an approach based on logic, calculation, and demonstration. The changes are not purely at the points of data collection and analysis, but extend to the social and economic organisation of science, with new categories of physicists emerging, people who are not quite experimenters and not quite theorists. Dyson [-@Dyson_2000; -@dyson2012science] has further explored this tool-driven approach, linking the Galilean revolution to the use of the telescope in astronomy, and the Crick-Watson revolution to the use of X-ray diffraction.

Galison’s emphasis on tool-driven change, as a complement to Kuhnian concept-driven change, has much potential for archaeology. The centrality of material culture in Galison’s view raises the question about whether there is scope for archaeologists to make productive contributions to understanding change in the history science through an archaeological analysis of scientific instrumentation [cf. @Schiffer_2013]. Furthermore, Galison’s analysis of the image/logic contrast in the history of physics invites a similar analysis of the history of archaeology Can we identify archaeological traditions focused on data collection and analysis using images and image-making devices [cf. @molyneaux2013cultural], in contrast to digital devices? One candidate for this might be the shift from building relative chronologies based on seriation of visually distinctive artefacts to absolute chronologies based on radiometric dating using computer-controlled instruments. A third issue is how we can use a tool-driven approach not just as a framework for understanding change in science, but as a method to predict or generate change in archaeology?

Bibliometric analysis of tool-driven change

The question of predicting or generating change in scientific practice via adoption of new tools is especially intruiging, and in this section we explore how a change in ecology might be relevant for understanding the future of archaeology. @Touchon_McCoy_2016 investigated evaluated changes in statistics used by ecologists as a potential area of tool-driven change. They searched nearly 20,000 published articles in ecology published between 1990 and 2013. They found that there has been a rise in complex and computationally intensive statistical techniques such as mixed effects models and Bayesian statistics and a decline in reliance on approaches such as ANOVA or t-tests. Crucially, they found that ecologists have shifted away from software tools such as SAS and SPSS to the open source program R.

@Touchon_McCoy_2016 identify four factors relating to technological change that might explain the changes they observed in the use of statistics in ecology. First, they note that automated data loggers, GPS trackers, remote sensing, and crowd sourcing have greatly increased rate at which ecologists collect data. Second, increases in desktop and cluster computing power has made complex analytical processes faster and more convienent to compute. Third, the developme of free, open source and easily extensible software such as R allows new methods to spread quickly via online fora and social media

there has been a rise in powerful desktop and supercomputing available to anyone in a university setting. Third, and partly as a result of the increase in computing power available, there has been an explosion in new analytical techniques (e.g., Pinheiro and Bates 2000, Gelman and Hill 2006, McCarthy 2007, Bolker 2008, Legendre and Legendre 2012). The rise of powerful, freely available, open source analytical software like R (R Development Core Team 2013) and associated wiki’s, blogs, and listservs where users consult with specialists and find code written by ecologists for ecologists allows the powerful combination of new methods and large data sets to be harnessed by us all. This process has democratized statistical inference, giving the average ecologist the power to conduct analyses previously reserved only for statisticians. Although the analyses that one can conduct in R are often achievable in a program like SAS, the community of ecologist statisticians using R has proliferated rapidly and may be driving change in techniques we document here (although it is admittedly impossible to tease apart cause and effect in this case). This community of users, who share code and advice about analyses, has enabled a great number of ecologists to conduct advanced analyses on their data. Indeed, the flexible nature of statistical programming is what allows those so inclined to “roll their own” statistics. Furthermore, the fact that R and other open source software is free has empowered a global community of users, including those in developing nations that may not have access to commercial costly programs. The fourth and final factor driving the change in statistical practices is that a number of seminal books and papers have fundamentally changed the way many people think about data analyses. To give just one example, Burnham and Anderson’s (2002) book on model selection and inference has been cited nearly 30,000 times since its publication in 2002 according to Google Scholar (website accessed 8 December 2015).

We explore this debate in light of recent efforts by many scientific disciplines to embrace methods to improve the reproducibility of their research. One of the most profound changes driven by this concern for reproducibility and transparency is from analysing data using tools dependent on pointing-and-clicking with a mouse in commercial software, to a new set of tools based on writing scripts in free and open source programming languages…

Tool-adoption in archaeology and elsewhere

We review bibliometric evidence for this change in the sciences generally and in archaeology in particular to test if the adoption of these new tools may be considered a revolution or some other process. We evaluate how computational approaches to improving reproducibility and transparency in archaeology are mediated and transformed by digital approaches.

Discussion

rrtools… huskydown

special role of archaeology here, since we are concerned about past behaviours, we have a special interest in research leaving a visible trace of our work

Conclusion

Acknowledgements

Our project comes from the group who took part in the 2017 Summer School on Reproducible Research in Landscape Archaeology at the Freie Universität Berlin (17-21 July), funded and jointly organized by Excellence Cluster 264 Topoi, the Collaborative Research Center 1266, and ISAAKiel.

Slide: Articles in all sciences citing R

This plot shows how data analysis is changing in the sciences more broadly towards the use of the R programming language. We searched the Web of Science, produced by Clarivate Analytics, for citations to the R program over the last ten years, and filtered the results to the top ten journals with the highest number of citations to this software. The purpose of this filtering step was to find areas of research were R is generally known, so we can see how its use is changing over time. Note that many articles that use R do not cite the R program, so these numbers underestimate the true use of R in science.

We can make two major observations:

First, the areas of science that have seen big growth in the use of R are ecology, evolution and related biological sciences. This is especially significant for us archaeologists, as ecology and biology are subjects from which we quite often adapt methods for analysis, as for example in point-pattern-analysis and statistical modelling. We also see increases in multi-disciplinary journals such as PeerJ and PLOS One.

Second, the scale of the increases is quite impressive: Starting only 2012 some journals now have 20% of articles citing R.

So, we can see this is a substantial change in the way researchers are working in many fields. What about archaeology? Do we also see a change like this?

Slide: Archaeology articles citing R

Here are the results for archaeological articles. We found 66 articles that cite R over the last ten years. Note that this is certainly not complete, since these are just the journals tracked by the Web of Science database, and we know that archaeologists publish in a great variety of journals.

There are three important details in these plots:

The first detail we see here is that overall, the proportions and the absolute number of articles citing R in archaeology are much smaller than what we see in the ecology journals on the previous slide. Only the Journal of Archaeological Science has more than 10 articles in our sample. It’s obvious, we have yet to adopt programming for data analysis in the same way that ecologists have.

The second detail is that we do not see a significant increase over time. Both archaeology and the other sciences we saw on the previous slide show an increase in citing R at around 2013-2014, but we do not yet see much of an upward trend in the archaeology journals. Indeed, in several of our journals the first citations of R only occured in the last two years. The plot on the upper right of this slide shows a statistical test of change over time for articles in JAS. With a low r-squared value and high p-value for the linear model, we conclude that there is no evidence for an increase in citations over time in this journal.

The third detail we see is the distinctive types of journals that have articles citing R. There is a clear focus on scientific archaeology, with Journal of Archaeological Science having the highest total number of articles. Archaeology as a subject has always been situated between a humanistic and a scientific tradition, but studies using statistical methods lean towards the sciences, which we mean with the term “scientific archaeology”. We also see geoarchaeology, lithics, and environmental archaeology represented in the journal names. We can investigate this further with statistical analysis of the words in the titles of all the articles in our sample.

Slide: What are archaeologists doing with R?

Here we see a comparison of word frequencies of the titles of archaeology journal articles. Words near the red line are used with about equal frequencies by papers that cite R and by papers that do not cite R. Words far away from the red line are used much more by one group of articles compared to the other.

This plot supports our initial observations: archaeologists using R tend to be doing more scientific archaeology. We see that many words relating to stone artefacts, zooarchaeology, and geoarchaeology are more frequently used by R-using archaeologists. On the other hand, articles that do not cite R more often discuss the Neolithic and metal-age periods, and less often reflect a technical specialisation.

Slide: The unrealised potential of R for reproducible research [signpost slide only]

After this overview how R is being used at the moment, we want to focus on the possibilities of making research reproducible using R.

Slide 9: The unrealised potential of R for reproducible research

We would now like to focus on those archaeological papers using R, that are demonstrating the potential of R for open and reproducible research. These papers are using R Markdown for their research, which is shown in this figure.

On the left we see a plain-text document written in R Markdown. This means that is has lines of code in amongst lines of normal text. We can format the text with simple additions, such as the underscore to indicate italics and the hash symbol to indicate a heading. These simple additions are Markdown, a very easy markup language with plain text formatting syntax. It is designed so that it can be converted to HTML and many other formats. R Markdown allows us to mix R code and text in the same document. Here we see a short snippet of R code right inside the text (nrow(cars)) as well as a longer block of code below.

When this R Markdown document is rendered, or converted to another format such as PDF or Microsoft Word, we see that the code is replaced by the output it generates. On the right side of this figure we see that the inline code has been replaced by the number 50, and the code block below has been replaced by the plot.

We have found that some archaeologists are writing their entire journal articles useing R Markdown, and they share their plain-text R Markdown document along with their publication. This means that the reader can see all of the details of the data analysis, and all of the small decisions made along the way. This way the analysis is easily reproducible for others.

Slide: Real-world example from Barton et al. 2018

Here is a real-world example R Markdown in use, from an archaeology article published in 2017.

Slide: An increasing number of archaeologists are publishing reproducible research

In our sample of R citing archaeology articles, we identified 14 articles that include R script- and data files, either in the supplementary materials or in a trustworthy data repository. This plot shows that although this number is small, it is increasing, suggesting the beginning of a new approach to how archaeologists share their research.

The way in which these 14 articles make their work reproducible is by:

  • making the data files available with their publication
  • making the R script files available with the publication
  • using R Markdown to add narrative text with the R code to give explanation and make it easier for readers to understand the analysis

Articles that have these three characteristics are part of a new paradigm in scientific communication, and with these 14 articles, we believe we may have identified the beginning of this behaviour in archaeology. We think it will not be hard for archaeologists that already use R to make the transition to producing reproducible research by sharing their code and data. If this can be achieved, then we can realise some of the potential that R has for making archaeology more open and reproducible.

Slide: Our new package, rrtools, and how it helps

As we studied the papers in our sample, we looked for common approaches to making and organising the code and data files. We found that there were few common standards used by archaeologists in making their work reproducible. There was a great variety of information presented in the code and data files. Some papers include code that will generate every figure and number in the text, while other papers only include code to produce one or two key figures in the paper.

Looking further afield, we also reviewed current efforts towards improving the reproducibility of research in other disciplines. In reviewing the literature we found several useful guides in diverse fields such as ecology and fluid dynamics. These guides described several best practices that are not yet common in archaeology.

So, after looking at the reproducible articles in archaeology, and the best practices in other disciplines, we wanted to make a simple toolkit that will help archaeologists to write reproducible research as easily as possible. This toolkit is an R package called rrtools. This package contains a small number of functions that automate many common analysis setup tasks.

On this slide we show the typical use of the key functions in rrtools that a researcher would use to quickly set up a compendium suitable for writing reproducible research in R. Each of these steps reflects a best practice that we’ve noted in our review of the literature. We will focus now on the first five:

  • use_compendium: This function creates a new directory for the research project, and creates an R package within this directory. We use the R package structure as the standard for our directory because it is widely recognised by R users. This means that people know where to look for code, where to look for data, and where to look for dependancies and other information.
  • usethis::use_mit_license: This function creates a copy of the MIT software lisence to the compendium. This is important for two reasons. First, because it tells the reader that the author is happy for their code to be reused by other people. And second, it tells the reader that the author does not take responsibility for any problems that the reader might have when they use the code. This is important for setting the expectations about the relationship between the author and the reader, regarding the use of code.
  • devtools::use_github: This will initiate the use of the Git version control system. This is the current state-of-the-art for version control of any kind of plain text file, including text, code, data, and images. This will also create a repository on the GitHub website. Using version control is important because it allows the author to time-travel back to earlier states of their analysis, if they take a dead-end path. It is also important for making collaboration smooth because many people can work on one set of files without losing track of what the most recent version is. Plus it is nice to have a remote back-up of our work in case of an emergency. The GitHub repository can be kept private if desired, until the work is published.
  • use_readme_rmd: This will create a simple document that is usually the first thing a reader sees when they browse the code files. This document is important because it helps to describe to the reader what to expect in the compendium, and it gives details of how to cite the compendium.
  • use_analysis: This function creates a set of folders according to the best practices for organising a typical reasearch project. We recognise that this file structure wont be perfect for every project. But we know that it will suit many projects quite well, and it will at least make the user think about how to organise their files logically, and according to best practises.

Slide: The best-practice directory structure from rrtools

Here is the file structure the rrtools gives us. For now, we want to point out two important principles that it enforces:

  • Keep the data separate from the methods. This means keep the code in a separate directory to the data
  • Keep the raw data seprate from data that is created or derived during the analysis.

These two fundamental principles make it easy to stay organised during a complicated project, and make it easier for a reader to navigate their way through the compendium.

Slide: Summary

To summarise:

  • We have shown a bibliometric analysis that reveals substantial change in the use of the R programming languages in the sciences generally. We saw that ecological and evolutionary sciences have really embraced R in their publications.

  • We have also shown that archaeologists have more recently taken up R, but to a much lesser degree, so far, than what we see in the other fields. We’ve seen that the use of R is most common in more scientific aspects of archaeology.

  • We also identified a small sub-set of archaeology papers that use R to make their work fully reproducible. There is a lot of potential for other archaeologists to make their work reproducible, and we think this would be a good thing for archaeology generally.

  • We have presented our freely available R package, rrtools, which is designed to make it easy for researchers in archaeology and other fields to make their research reproducible. We hope this will help to realise some of the potential to make archaeological research more open and reproducible.

Thank you.

=======

There is an argument among philosophers of science that revolutions in science are either ideas-driven or tool-driven. One of the most widely-known models of ideas-driven change in science is Kuhn’s (1962) effort to describe the history of science by proposing paradigm shifts resulting from revolutionary change in communities of researchers. In brief, this model proposes that the history of science consists of long periods of tradition-bound ‘normal science’ punctuated by short episodes of revolution. Normal science was described by Kuhn as stagnant, routine day-to-day research focused on ‘puzzle solving’. What counts as suitable puzzles, and acceptable solutions to these puzzles, is governed by the norms and procedures of the prevailing paradigm. Revolutions in science occur as punctuations in the equilibria of normal science, when anomalous observations culminate in changes that replace one paradigm with another in a conversion or speciation process where the community adopts a new way of doing science. There has been substantial debate about how to draw a clear distinction between so-called normal and revolutionary science. (Watkins 1970; Toulmin 1970; Casadevall and Fang 2016). A notable criticism is that Kuhn’s normal science is unscientific, a situation where critical science had contracted into defensive metaphysics resulting from the domination of a ruling dogma (Popper 1970). This is at odds with a view of science that it is in a continuous state of revolution, with researchers simultaneously employing several styles of thinking and doing science, and continuously confronting theory with evidence and modifying their ideas based on the outcome. In this view, based on the post-World War II era of science, radical discontinuities are rare, and major developments instead emerge from the division and recombination of already mature fields (Andersen 2013). Given the rarity of radical discontinuities, some biologists have argued that change in biology is better characterised by new ideas (e.g. Mendelian heredity, Darwinian evolution, and molecular genetics) replacing not a former paradigm but a conceptual vacuum (Wilkins 1996).

In archaeology we have seen claims for paradigm shifts in several areas. The most wide-ranging application is Clark’s (1993) comparison of North American archaeology to European archaeology during the 1970s-90s. Clark, similar to Binford and Sabloff (1982), identified the North American archaeological paradigm as rooted in anthropology, founded on post-positivist biases, and distinctive with it critical-realist ontology, a modified-objectivist epistemology, and an experimental-manipulative methodology. In contrast, the European archaeological paradigm is described a kind of history, mostly in a strict empiricist tradition, and typified by a realist ontology, an inductivist epistemology, and by an observational methodology. Clark’s analysis describes a multi-paradigm situation where multiple research communities (i.e. North American and European) simutaneously operate in different paradigms (which Kuhn noted as a sign of a developing or immature discipline). His account is not a complete Kuhnian analysis because it lacks a revolutionary change event where one paradigm is replaced by another, although he hints that processual archaeology may be considered a radical discontinuity relative to the culture-historical approaches that preceeded it (Clark 1993, 206).

A more thorough treatment of the claim that processual archaeology was a paradigm shift is provided by Meltzer (1979). Meltzer reviews literature arguing for a revolutionary change in archaeology during the 1960s and 1970s, and considers these claims in light of classic Kuhnian revolutionary events such as the replacement of the Ptolemaic system by the Copernican system, and Newtonian dynamics by the new Einsteinian dynamics. He finds that the changes that occured in Anglophone archaeology after the 1960s were incremental, and primarily of method, rather than a widespread replacement of one ontological structure by another one, incommensurable or incompatible with the former. Trigger (2006, 538) came to a similar conclusion at the end of his broad survey of the history of archaeology, that changes throughout archaeology have been mostly additive with only partial replacement.

We see less formal mentions of paradigm shifts in archaeology in reference to processual to post-processual archaeology (Härke 2002), as well as more localised and specific shifts in archaeological practice and thinking. For example, Snodgrass (2002) claims that among Classical Archaeologists, an increase in research on previously neglected periods odf antiquity, such as the Greek Early Iron Age, constitutes a paradigm shift. McAnany and Rowe (2015) propose that the appearance of community-based participatory models of research among some communities of archaeologists is a paradigm shift, though they concede that it is more transformational rather than revolutionary. Fuller (2010) has used the term to describe a shift in thinking about the emergence of agriculture as a parallel, simultaneous event at the start of the Holocene (the Neolithic Revolution), to new concept of the emergence as a protracted and entangled process, happening 20 or so times independently, following different pathways. Most of these paradigms are quite remote in meaning from Kuhn’s classical formulation, and this highlights an important complication with using Kuhn’s paradigm shift approach, both as a tool to understand the history of science and as a guide to changing it, which is Kuhn’s multiple definitions of a paradigm.

Masterman (1970) has documented 21 different uses by Kuhn, which she organises into three groups. First is the metaphysical notion of a set of beliefs, and this is the sense that dominates most of the critical writing on Kuhn’s model. Second is the sociological sense of scientific habits, the universially recognised scientific achievements that are the foundations of day-to-day normal science. These include specific attention-grabbing successes that provide a touchstone for the coordination of future research. Third is the concrete sense of an actual textbook, instrumentation or toolkit. This recognition of the multivalent nature of paradigms is important because it broadens the locus for scientific revolution to include sociological practices of science, such as might be produced by changes in norms of publication and peer review, and to include the physical infrastructure of doing science, such as the technology for collecting and analysing data, and for communication and collaboration among researchers.

We can see a compelling exploration of the concrete sense of a paradigm in the work of Galison (1997), who has offered a tool-driven view of change in research communities. Galison analysed of the role of tools in twentieth century particle physics, starting with hand-crafted cloud chambers and bubble chambers and ending with digital counters of and computers. He identifies a profound change in physics when analog devices producing pictures were superseded by digital devices producing numerical data. At the heart of this change is a shift from an intuitive approach, stimulated by visual and pictorial model-building to an approach based on logic, calculation, and demonstration. The changes are not purely at the points of data collection and analysis, but extend to the social and economic organisation of science, with new categories of physicists emerging, people who are not quite experimenters and not quite theorists. Dyson (2000, 2012) has further explored this tool-driven approach, linking the Galilean revolution to the use of the telescope in astronomy, and the Crick-Watson revolution to the use of X-ray diffraction.

Galison’s emphasis on tool-driven change, as a complement to Kuhnian concept-driven change, has much potential for archaeology. The centrality of material culture in Galison’s view raises the question about whether there is scope for archaeologists to make productive contributions to understanding change in the history science through an archaeological analysis of scientific instrumentation (cf. Schiffer 2013). Furthermore, Galison’s analysis of the image/logic contrast in the history of physics invites a similar analysis of the history of archaeology Can we identify archaeological traditions focused on data collection and analysis using images and image-making devices (cf. Molyneaux 2013), in contrast to digital devices? One candidate for this might be the shift from building relative chronologies based on seriation of visually distinctive artefacts to absolute chronologies based on radiometric dating using computer-controlled instruments. A third issue is how we can use a tool-driven approach not just as a framework for understanding change in science, but as a method to predict or generate change in archaeology?

Bibliometric analysis of tool-driven change

The question of predicting or generating change in scientific practice via adoption of new tools is especially intruiging, and in this section we explore how a change that has been observed in ecology might be relevant for understanding or even directing the future of archaeology.

Looking back on a tool-driven change in Ecology

Touchon and McCoy (2016) investigated evaluated changes in statistics used by ecologists as a potential area of tool-driven change. They searched nearly 20,000 published articles in ecology published between 1990 and 2013. They found that there has been a rise in complex and computationally intensive statistical techniques such as mixed effects models and Bayesian statistics and a decline in reliance on approaches such as ANOVA or t-tests. Crucially, they found that ecologists have shifted away from software tools such as SAS and SPSS to the open source program R.

Touchon and McCoy (2016) identify four factors relating to technological change that might explain the changes they observed in the use of statistics in ecology. First, they note that automated data loggers, GPS trackers, remote sensing, and crowd sourcing have greatly increased rate at which ecologists collect data. Second, increases in desktop and cluster computing power has made complex analytical processes faster and more convienent to compute. Third, the development of free, open source and easily extensible software for data analysis and visualisation, such as R, allows new methods to spread quickly via online fora and social media. Fourth, the publication of several books and papers that have strongly influenced the way many ecologists think about data analysis, such as Burnham and Anderson (2003)’s book on model selection and inference.

While all of these four factors could be considered relevant to the tooling of ecology and archaeology, we focus here on the third one because of its relevance to archaeology. Archaeology and ecology each have a high diversity of data collection methods, and types of data analyses they conduct, with only a small overlap. What they share in common is working with the field-collected data on a computer to prepare it for publication. R is the most common open source and free ware data analysis tool available and therefore a good proxy for the adoption of open source scientific programming languages in a discipline. To investigate changes in the use of R in archaeology and ecology, we obtained reference lists from a sample of scholarly articles in the Web of Science database and examined patterns in the citation of the R program over time.

We used the ‘Cited Reference Search’ function provided by the Web of Science online scientific citation indexing service to identify journal articles that cite R. Although R has been available since the late 1990s (Thieme 2018), a recommended format for citing the software did not appear until 2004, with the author given as “R Development Core Team”. This recommended format for citing the software changed slightly in 2012 when the author was updated to “R Core Team”. We searched the Web of Science using ‘“R DEV COR TEAM” OR “R CORE” OR “R CORE TEAM” OR “R DEVELOPMENT CORE TEAM”’ in the CITED AUTHOR field. We sorted the results by frequency of citation, and selected the first 1000 items (the maximum allowed by the Web of Science service). These 1000 items represent variations on the recommended format for citing R, and found in the reference lists of 33,326 English-languge articles indexed by the Web of Science in the research area of Environmental Sciences Ecology. We then downloaded the bibliographic data for all of these articles.

Figure ?? shows the percentage of articles citing R in each of several of the top ecology journals (as defined by how often their articles are cited). We restrict the start of the observation period to 2007 for convenience. The plot shows that the percentage of articles citing R has increased from less than ten percent in all journals in the late 2000s, to more than 30% in Ecosphere, Ecology and Evolution, and Molecular Ecology after 2012. We would not call this a Kuhnian paradigm shift, but it does show a substantial change in the tools of the discipline, supporting claims for a Galisonian tool-driven change in ecology similar to the changes described by Touchon and McCoy (2016).

Percentage of articles per year citing R in top Ecology journals (3,709 articles out of 33,326). Data from Web of Science for 2007-2017

Figure 1: Percentage of articles per year citing R in top Ecology journals (3,709 articles out of 33,326). Data from Web of Science for 2007-2017

Looking forward to a tool-driven change in Archaeology

To compare with ecology we conducted a similar bibliometric analysis for archaeology journals indexed by the Web of Science service. We used the ‘Cited Reference Search’ to find articles citing R in the same way as above, and then refined the results to keep only those articles published during 2007-2017 that are included in the Web of Science category ‘Archaeology’. This resulted in 42443 articles, of which 116 cite R. Figure ?? shows the temporal trend of citations of R in archaeological articles. There are three interesting details revealed by this figure.

Percentage of Archaeology articles per year citing R (116 out of 42,443 articles in sample. Data from Web of Science for 2007-2017

Figure 2: Percentage of Archaeology articles per year citing R (116 out of 42,443 articles in sample. Data from Web of Science for 2007-2017

The first detail we see here is that overall, the proportions and the absolute number of articles citing R in archaeology are much smaller than what we see in the ecology journals in Figure ??. Only the Journal of Archaeological Science has more than 50 articles in our sample. We might conclude from this observation that archaeologists have yet to adopt programming for data analysis and visualisation in the same way that ecologists have.

The second detail is that most archaeology journals do not show any increase in the percentage of article citing R over time . The ecology journals shown in Figure ?? show clear upward trends of increasing proportions of articles citing R over time, but we do not see any clear trends in the archaeology articles considered together. Indeed, in several of the archaeological journals in our sample, the first citations of R only occured in the last two years. The sample sizes for the archaeology journals are too small to confidently infer any trend of over time, with the exception of the Journal of Archaeological Science with 62 articles. The inset plot in Figure ?? shows a statistical test of change over time for articles in the Journal of Archaeological Science. With a moderate r-squared value and low p-value for the linear model, we can conclude that there is some evidence for an increase in citations of R over time in this journal. This may indicate an increasing use of R among archaeologists, especially those working at the intersection of the natural and physical sciences. This may hint at the start of a trend similar to what we see in the ecology journals of a widespread adoption of R.

Following on from this observation about the increasing popularity of R in the Journal of Archaeological Science, the third detail we see in these results is the distinctive types of journals that have articles citing R. There is a clear focus on journals that publish scientific archaeology, especially those that focus on specialised empirical analysis. In addition to the general archaeology journals, we also see Lithic Technology, Anthropozoologica, and International Journal of Osteology in the journal names, indicating that we find R used by specialists in stone artefacts and faunal remains. To further investigate the topics that archaeologists are using R to study, we conducted a statistical analysis of the words in the titles of all the articles in our sample.

What are R-using archaeologists writing about?

We computed a comparison of word frequences in journal article titles in the Web of Science data to get a better understanding of what topics archaeologists are writing about when they cite R. First we separated the archaeology articles into two groups, those that cite R, and those that do not. Second, we filtered to keep only words that occur in titles in both groups, and removed stop words that are very common and uninformative (e.g. ‘the’, ‘archaeological’, ‘study’). This resulted in 41,236 words in 42451. Third, for each word found in all the titles of the journal articles in each group, we computed its proportion of the total number of words in all the titles in each group. Figure ?? shows the results of this analysis. Words near the red line are used with about equal frequencies by papers that cite R and by papers that do not cite R. Words far away from the red line are used much more by one group of articles compared to the other.

Comparing the frequency of words used in titles of archaeology articles (41,236 words in 42,451 articles)

Figure 3: Comparing the frequency of words used in titles of archaeology articles (41,236 words in 42,451 articles)

Figure 3 supports our initial observations that archaeologists using R tend to be doing archaeology that involves the natural and physical sciences. Articles that cite R more often have titles that include terms such as ‘lithic’, ‘experimental’, and ‘isotope’. On the other hand, articles that do not cite R more often discuss the Roman and Bronze Age periods, and less often reflect a technical specialisation, if there is one, it seems to be in ceramics.

From using code in research to sharing code with publications

Touchon and McCoy (2016) had identified the use of scripting languages as being a factor for easily sharing methods and code, thereby increasing the rate of development of statistical methods. One notable practice that we observed during our bibliometric analysis is that among authors of articles citing R, a subset of these authors make publicly available the code and data that they used to generate the results presented in their article. We identified 35 articles that include R script and data files, either in the supplementary materials or in a trustworthy data repository. Our contiunuously updated list of articles in this sample is available online here: https://github.com/benmarwick/ctv-archaeology. Figure ?? shows that although this number is small, it is increasing, hinting at the emergence of a new approach to how archaeologists share their research.

These archaeology articles that include publicly available R code and data are part of a new approach to scientific communication that has also been observed in other fields. Although we are not aware of attempts to quantify this change in the same way we have for archaeology, we have observed a number of recent publications that describe and recommend code sharing in statistics (Baumer et al. 2014), genome biology (Markowetz 2015), computational biology (Sandve et al. 2013), biostatistics (Peng 2009), computer science (Peng 2011; Mitchell, LeVeque, and Stodden 2012), applied mathematics (LeVeque, Mitchell, and Stodden 2012), speech science (Abari 2012), political science [King (1995); Dafoe2014], and the social sciences generally (Miguel et al. 2014). As part of this growing interest in using code for research we also see manifestos aimed at researchers doing any kind of quantitative work (Nosek et al. 2015; Ince, Hatton, and Graham-Cumming 2012; Barnes 2010) and articles recommending using R in undergraduate education (Bray, Çetinkaya-Rundel, and Stangl 2014; Eglen 2009). Much of this literature is concerned with identifying and solving problems of irreproducibility in research. This comes from a renewed interest in basic principles of the scientific method: that once-off results should not be trusted, instead they should be reproducible by other members of the research community. Furthermore, an article should sufficiently describe the presented research results such that a colleague can fully understand the results and how they were obtained. Many of these authors trace problems of irreproducibility to the increased complextity of computer-based analyses (but only a limited space to describe them in a journal article), and the use of mouse-driven computer programs where the researcher’s analytical decisions are not recorded during the research process, and thus cannot be shared with other researchers.

In our view, merely using R (or similar open source programming languages) to conduct research might not constitute a tool-driven revolutionb by itself, but publicaly sharing the code used for research, as a solution to problems of irreproducibility, could enable revolutionary change. Openly shared code could speed the widespread adoption of the reproducibility are a core tenet of the scientific process, since it frees researchers from the black box that most mouse-driven computers are, and enables researchers to not only rerun the shared analysis, but to gain access to all parameter settings, empowering them to change these and therefore properly evaluate, extend and reuse the published results. As open source languages such as R (or Python…) are free to use, and most trustworthy repositories are also free (such as zenodo, osf, figshare, etc.), then even researchers with limited resources anywhere in the world can to contribute equally to the research community by using and sharing code.

Articles in archaeology journals using R for reproducible research (n = 35)

Figure 4: Articles in archaeology journals using R for reproducible research (n = 35)

A pragamtic issue relating to wide-spread changes in scientific practice is what the incentives are for members of the research community. This is especially a concern for researchers that might not be motivated by a desire to adhere to the abstract basic principles of what it means to practice science with integrity. If code sharing has some concrete career advantages then we might expect it to be more likely to be part of a tool-drive revolution. Vandewalle (2012) analysed citation rates for 645 articles published in IEEE Transactions on Image Processing during 2004–2006. He found statistically significant differences between the median number of citations for papers that have code available and papers that do not have code available. In Figure 5 we conducted a similar anlysis for archaeology articles using the ‘TC’ field in the data we obtained from the Web of Science service. TC stands for ‘Web of Science Core Collection Times Cited Count’. This Times Cited count displays the total number of times a published paper was cited by other papers within the Web of Science Core Collection. Unlike Google Scholar, this does not include citations on blogs, pre-prints and other places outside of journal articles, so the citation counts from Web of Science are lower than those on Google Scholar. We see that archaeological articles that cite R are consistently more highly cited themselves than articles that do not cite R.

Median citation rates per year for archaeology articles that cite R (n = 116) and articles that do not cite R (n = 42,335)

Figure 5: Median citation rates per year for archaeology articles that cite R (n = 116) and articles that do not cite R (n = 42,335)

Discussion

In light of the discussion in the philosophy of science, whether paradigm-shifts are tool- or idea-driven, we offer the hypothesis that a tool-driven change of paradigms is underway in the biological sciences and, more subtly, in archaeology. The tool which drives this change is the use of scripting languages in computational approaches of these disciplines. We show that the use of R is on the rise among archaeologists using natural and physical science methods, and has concrete advantages for authors that publish papers that cite R as they tend to be referenced more often, giving the authors more visiblity and influence. This also is an indicator for the increase in popularity of R.

The use of R or any other scripting languages enables a major shift in the technical and social dynamics of the research community, as it offers the possibility to share one’s code. The sharing of code has been identified in other disciplines as an amplifier of change in methods(???). It also enables researchers to not just reproduce the work of others, but to also evaluate them according to the scientific method. Thus, the sharing of code may be at the root of an upcoming tool-driven change for archaeology, consistent with what we see desribed in the work of (???; 1997). This change does not take place revolutionary-quick and does not replace older ideas as such (as has been a critique of Kuhn’s idea-driven concept of paradigm shift by others (???, @trigger_2006)).

Rather, this change enables archaologists to more faithfully adhere to the fundamental tenet of the scientific process which demands the reproducibility of results. We do not want to claim archaeologists did not want to create reproducible research before the advance of scientific scripting languages. But instead that with the introduction of computers there has been a gradual and unintended drift away from reproducibility, and thus from scientific practice. By using and sharing code we can undo this drift, and return to a robust scrietific practice for archaeology.

There is a longstanding tradition in archaeology of publishing catalogues, lists and maps showing the data used for analyses. In the digital age, though, many archaeological tasks have shifted from the descriptive and qualitative aspects to quantitative and computational analyses. This shift also mandates a change in documentation of analyses to enable other researchers to reproduce published results. Modern archaeology has often been lacking in this aspect, as the research has been undertaken using point-and-click software, which obscures and complicates the reproducibility of analyses. As we show, the advance of scripting languages takes place mostly in the part of archaeology that is in close contact with natural and physical sciences.

We do not believe that a Kuhnian paradigm-change seems to be happening in ecology or archaeology: Data-and method sharing behaviour has been around in archaeology before and do not present incommensurate ways of thinking about the subject. What we can see, though, is that the need for methods to share research in a way that is reproducible for others has been acknowledged and that archaeologists have looked for tools to make this possible. Using scripting languages and data repositories to share the scripts is a solution that we have demonstrated enjoys increasing popularity in the archaeological research community.

Making it easier

We acknowledge that learning to use code, if one has not recieved formal instruction, is not an easy task. Similarly, sharing code in public takes a little time to ensure that it is fit for others to read. Our casual observations of the ways in which archaeologists are using and sharing code show a high degree of variability in code style and organisation, indicating that currently most archaeologists are independantly solving problems of how to write and share code through trial and error, rather than following a model or widely-used approach. This suggests to us that there is some scope to improve the ease and efficiency of code-use in archaeology by studying the exsiting practice and providing a model that others can easily adapt to their own research projects.

The result of our analysis of existing practices among archaeologists and other researchers, and our study of best practices in scientific computing (described in more detail in Marwick, Boettiger, and Mullen (2018)) is the R package rrtools (‘reproducible research tools’, https://github.com/benmarwick/rrtools). The goal of rrtools ist o make it easier for archaeologists and researchers in other fields to use R for research and publication. This package aims to simplify many of the steps needed to write reproducible research, and to guide users to best practices with minimal effort. The rrtools package contains functions that create a file structure according to fundamental principles of organising files for research. First, we keep the data separate from the methods. This means keep the code in a separate directory to the data. Second, we, keep the raw data seprate from data that is created or derived during the analysis.These two basic principles make it easy to stay organised during a complicated project, and make it easier for a reader to navigate their way through the compendium of files that are shared with the publication.

The rrtools package contains several functions for the R user to enable a researcher to quickly set up a compendium suitable for writing reproducible research in R. Each of these steps reflects a best practice that has been previously articulated in other feilds. We will focus here on the first five, as these are most relevant to most researchers:

  • rrtools::use_compendium: This function creates a new directory for the research project, and creates an R package within this directory. We use the R package structure as the standard for our directory because it is widely recognised by R users. This means that people know where to look for code, where to look for data, and where to look for dependancies and other information.
  • usethis::use_mit_license: This function creates a copy of the MIT software lisence to the compendium. This is important for two reasons. First, because it tells the reader that the author is happy for their code to be reused by other people. And second, it tells the reader that the author does not take responsibility for any problems that the reader might have when they use the code. This is important for setting the expectations about the relationship between the author and the reader, regarding the use of code.
  • usethis::use_github: This will initiate the use of the Git version control system. This is the current state-of-the-art for version control of any kind of plain text file, including text, code, data, and images. This will also create a repository on the GitHub website. Using version control is important because it allows the author to time-travel back to earlier states of their analysis, if they take a dead-end path. It is also important for making collaboration smooth because many people can work on one set of files without losing track of what the most recent version is. Plus it is nice to have a remote back-up of our work in case of an emergency. The GitHub repository can be kept private if desired, until the work is published.
  • rrtools::use_readme_rmd: This will create a simple document that is usually the first thing a reader sees when they browse the code files. This document is important because it helps to describe to the reader what to expect in the compendium, and it gives details of how to cite the compendium.
  • rrtools::use_analysis: This function creates a set of folders according to the best practices for organising a typical reasearch project. We recognise that this file structure wont be perfect for every project. But we know that it will suit many projects quite well, and it will at least make the user think about how to organise their files logically, and according to best practises.

Conclusion

Starting with the discussion of Kuhn’s influential work on paradigm-shifts in science [citation needed], we offer a second explanation on how pardigm-shifts develop in science with the help of Galison (???), who focuses on the material culture of a science and its tools. A change in the paradigms of a discipline may also only occur, if tools to implement this change are available. The change we are aiming for is the move to a reproducible computational archaeology, that adheres to the scientific method. The tool which we have been discussing here is R, as an example for scientific scripting languages.

We have shown a bibliometric analysis that reveals substantial change in the use of the R programming languages in ecology. We saw that ecological and evolutionary sciences have strongly embraced R in their publications. Archaeologists have more recently taken up R, but to a much lesser degree, so far, than what we see in the other fields, most commonly in more scientific aspects of archaeology. We also identified a small sub-set of archaeology papers that use R to make their work fully reproducible and could identify an advantage of citing R for researchers as it leads to being referenced more often.

There is great potential for archaeologists to make their work reproducible using R and related tools, and we think this is a much needed shift in computational archaeology - linking the digital and computational times to older practices of sharing data and methods. We therefore presented our freely available R package, rrtools, which is designed to make it easy for researchers in archaeology and other fields to make their research reproducible. We hope this will help to realise some of the potential to make computational archaeological research more open and reproducible speed the transfer of new results and methods throughout the research community without barriers due to resource availabilty, and in this way encourage researchers to adhere to the scientific process.

Acknowledgements

Our project comes from the group who took part in the 2017 Summer School on Reproducible Research in Landscape Archaeology at the Freie Universität Berlin (17-21 July), funded and jointly organized by Excellence Cluster 264 Topoi, the Collaborative Research Center 1266, and ISAAKiel.

>>>>>>> a4f724d0933cfc4b2322ddb3b951beb784e55322
pagebreak

References

<<<<<<< HEAD =======

Abari, Kálmán. 2012. “Reproducible Research in Speech Sciences.” International Journal of Computer Science Issues 9 (6): 43–52. http://www.ijcsi.org/papers/IJCSI-9-6-2-43-52.pdf.

Andersen, Hanne. 2013. “The Second Essential Tension: On Tradition and Innovation in Interdisciplinary Research.” Topoi 32 (1). Springer: 3–8.

Barnes, Nick. 2010. “Publish Your Computer Code: It Is Good Enough.” Nature News 467 (7317): 753–53. https://doi.org/10.1038/467753a.

Baumer, Ben, Mine Cetinkaya-Rundel, Andrew Bray, Linda Loi, and Nicholas J. Horton. 2014. “R Markdown: Integrating a Reproducible Analysis Tool into Introductory Statistics.” Technology Innovations in Statistics Education 8 (1). http://www.escholarship.org/uc/item/90b2f5xh.

Binford, Lewis R, and Jeremy A Sabloff. 1982. “Paradigms, Systematics, and Archaeology.” Journal of Anthropological Research 38 (2). University of New Mexico: 137–53.

Bray, Andrew, Mine Çetinkaya-Rundel, and Dalene Stangl. 2014. “Taking a Chance in the Classroom: Five Concrete Reasons Your Students Should Be Learning to Analyze Data in the Reproducible Paradigm.” Chance 27 (3): 53–56. https://doi.org/10.1080/09332480.2014.965635.

Burnham, Kenneth P, and David R Anderson. 2003. Model Selection and Multimodel Inference: A Practical Information-Theoretic Approach. Springer Science & Business Media.

Casadevall, Arturo, and Ferric C. Fang. 2016. “Revolutionary Science.” mBio 7 (2). American Society for Microbiology. https://doi.org/10.1128/mBio.00158-16.

Clark, G. A. 1993. “Paradigms in Science and Archaeology.” Journal of Archaeological Research 1 (3): 203–34. https://doi.org/10.1007/BF01326535.

Dyson, Freeman. 2000. The Sun, the Genome, and the Internet. New York Public Library Lectures in Humanities. Oxford University Press.

Dyson, Freeman J. 2012. “Is Science Mostly Driven by Ideas or by Tools?” Science 338 (6113). American Association for the Advancement of Science: 1426–7.

Eglen, Stephen J. 2009. “A Quick Guide to Teaching R Programming to Computational Biology Students.” PLoS Comput Biol 5 (8): e1000482. https://doi.org/10.1371/journal.pcbi.1000482.

Fuller, Dorian Q. 2010. “An Emerging Paradigm Shift in the Origins of Agriculture.” General Anthropology 17 (2): 1–12. https://doi.org/10.1111/j.1939-3466.2010.00010.x.

Galison, Peter. 1997. Image and Logic: A Material Culture of Microphysics. University of Chicago Press.

Härke, Heinrich. 2002. “Interdisciplinarity and the Archaeological Study of Death.” Mortality 7 (3): 340–41. https://doi.org/10.1080/1357627021000025487.

Ince, Darrel C., Les Hatton, and John Graham-Cumming. 2012. “The Case for Open Computer Programs.” Nature 482 (7386): 485–88. https://doi.org/10.1038/nature10836.

King, Gary. 1995. “Replication, Replication.” PS: Political Science & Politics 28 (03): 444–52.

Kuhn, T. 1962. The Structure of Scientific Revolutions. University of Chicago Press.

LeVeque, Randall J., Ian M. Mitchell, and V. Stodden. 2012. “Reproducible Research for Scientific Computing: Tools and Strategies for Changing the Culture.” Computing in Science & Engineering 14 (4): 13–17. https://doi.org/10.1109/MCSE.2012.38.

Markowetz, Florian. 2015. “Five Selfish Reasons to Work Reproducibly.” Genome Biology 16. BioMed Central.

Marwick, Ben, Carl Boettiger, and Lincoln Mullen. 2018. “Packaging Data Analytical Work Reproducibly Using R (and Friends).” The American Statistician 72 (1). Taylor & Francis: 80–88.

Masterman, Margaret. 1970. “The Nature of a Paradigm.” In Criticism and the Growth of Knowledge: Volume 4: Proceedings of the International Colloquium in the Philosophy of Science, London, 1965, edited by Alan Musgrave and others, 4:59–89. Cambridge University Press.

McAnany, Patricia A., and Sarah M. Rowe. 2015. “Re-Visiting the Field: Collaborative Archaeology as Paradigm Shift.” Journal of Field Archaeology 40 (5): 499–507. https://doi.org/10.1179/2042458215Y.0000000007.

Meltzer, David J. 1979. “Paradigms and the Nature of Change in American Archaeology.” American Antiquity 44 (4): 644–57. https://doi.org/10.2307/279104.

Miguel, E., C. Camerer, K. Casey, J. Cohen, K. M. Esterling, A. Gerber, R. Glennerster, et al. 2014. “Promoting Transparency in Social Science Research.” Science (New York, N.Y.) 343 (6166): 30–31. https://doi.org/10.1126/science.1245317.

Mitchell, Ian M., Randall J. LeVeque, and V. Stodden. 2012. “Reproducible Research for Scientific Computing: Tools and Strategies for Changing the Culture.” Computing in Science and Engineering 14 (4): 13–17.

Molyneaux, Brian Leigh. 2013. The Cultural Life of Images: Visual Representation in Archaeology. Routledge.

Nosek, BA, G Alter, GC Banks, D Borsboom, SD Bowman, SJ Breckler, S Buck, et al. 2015. “Promoting an Open Research Culture: Author Guidelines for Journals Could Help to Promote Transparency, Openness, and Reproducibility.” Science (New York, NY) 348 (6242). NIH Public Access: 1422.

Peng, Roger D. 2009. “Reproducible Research and Biostatistics.” Biostatistics 10 (3): 405–8. https://doi.org/10.1093/biostatistics/kxp014.

———. 2011. “Reproducible Research in Computational Science.” Science (New York, Ny) 334 (6060): 1226.

Popper, Karl. 1970. “Normal Science and Its Dangers.” In Criticism and the Growth of Knowledge: Volume 4: Proceedings of the International Colloquium in the Philosophy of Science, London, 1965, edited by Alan Musgrave and others, 4:51–58. Cambridge University Press.

Sandve, Geir Kjetil, Anton Nekrutenko, James Taylor, and Eivind Hovig. 2013. “Ten Simple Rules for Reproducible Computational Research.” PLoS Comput Biol 9 (10): e1003285. https://doi.org/10.1371/journal.pcbi.1003285.

Schiffer, Michael Brian. 2013. The Archaeology of Science: Studying the Creation of Useful Knowledge. Manuals in Archaeological Method, Theory and Technique. Springer International Publishing. //www.springer.com/us/book/9783319000763.

Snodgrass, Anthony. 2002. “A Paradigm Shift in Classical Archaeology?” Cambridge Archaeological Journal 12 (2): 179–94. https://doi.org/10.1017/S0959774302000094.

Thieme, Nick. 2018. “R Generation.” Significance 15 (4): 14–19. https://doi.org/10.1111/j.1740-9713.2018.01169.x.

Touchon, Justin C., and Michael W. McCoy. 2016. “The Mismatch Between Current Statistical Practice and Doctoral Training in Ecology.” Ecosphere 7 (8): e01394. https://doi.org/10.1002/ecs2.1394.

Toulmin, Stephen. 1970. “Does the Distinction Between Normal and Revolutionary Science Hold Water.” In Criticism and the Growth of Knowledge: Volume 4: Proceedings of the International Colloquium in the Philosophy of Science, London, 1965, edited by Alan Musgrave and others, 4:39–48. Cambridge University Press.

Trigger, Bruce G. 2006. A History of Archaeological Thought. 2nd ed. Cambridge University Press. http://gen.lib.rus.ec/book/index.php?md5=7F40DA33B758EF5685ED18C5A8F9476A.

Vandewalle, Patrick. 2012. “Code Sharing Is Associated with Research Impact in Image Processing.” Computing in Science and Engineering 14 (4): 42–47.

Watkins, John WN. 1970. “Against ‘Normal Science’.” Criticism and the Growth of Knowledge. Cambridge University Press Cambridge, 25–37.

Wilkins, Adam S. 1996. “Are There ‘Kuhnian’ Revolutions in Biology?” BioEssays 18 (9). Wiley Online Library: 695–96. https://doi.org/10.1002/bies.950180902.

>>>>>>> a4f724d0933cfc4b2322ddb3b951beb784e55322
pagebreak

Colophon

<<<<<<< HEAD

This report was generated on 2018-09-05 15:45:17 using the following computational environment and dependencies:

#>  setting  value                       
#>  version  R version 3.5.1 (2018-07-02)
#>  system   x86_64, mingw32             
#>  ui       RTerm                       
#>  language (EN)                        
#>  collate  German_Germany.1252         
#>  tz       Europe/Berlin               
#>  date     2018-09-05                  
#> 
#>  package      * version date       source        
#>  acepack        1.4.1   2016-10-29 CRAN (R 3.5.1)
#>  assertthat     0.2.0   2017-04-11 CRAN (R 3.5.1)
#>  backports      1.1.2   2017-12-13 CRAN (R 3.5.0)
#>  base         * 3.5.1   2018-07-02 local         
#>  base64enc      0.1-3   2015-07-28 CRAN (R 3.5.0)
#>  bindr          0.1.1   2018-03-13 CRAN (R 3.5.1)
#>  bindrcpp     * 0.2.2   2018-03-29 CRAN (R 3.5.1)
#>  bookdown       0.7     2018-02-18 CRAN (R 3.5.1)
#>  broom          0.5.0   2018-07-17 CRAN (R 3.5.1)
#>  cellranger     1.1.0   2016-07-27 CRAN (R 3.5.1)
#>  checkmate      1.8.5   2017-10-24 CRAN (R 3.5.1)
#>  cli            1.0.0   2017-11-05 CRAN (R 3.5.1)
#>  cluster        2.0.7-1 2018-04-13 CRAN (R 3.5.1)
#>  codetools      0.2-15  2016-10-05 CRAN (R 3.5.1)
#>  colorspace     1.3-2   2016-12-14 CRAN (R 3.5.1)
#>  compiler       3.5.1   2018-07-02 local         
#>  crayon         1.3.4   2017-09-16 CRAN (R 3.5.1)
#>  data.table     1.11.4  2018-05-27 CRAN (R 3.5.1)
#>  datasets     * 3.5.1   2018-07-02 local         
#>  devtools       1.13.6  2018-06-27 CRAN (R 3.5.1)
#>  digest         0.6.16  2018-08-22 CRAN (R 3.5.1)
#>  dplyr        * 0.7.6   2018-06-29 CRAN (R 3.5.1)
#>  evaluate       0.11    2018-07-17 CRAN (R 3.5.1)
#>  fansi          0.3.0   2018-08-13 CRAN (R 3.5.1)
#>  forcats      * 0.3.0   2018-02-19 CRAN (R 3.5.1)
#>  foreign        0.8-70  2017-11-28 CRAN (R 3.5.1)
#>  Formula      * 1.2-3   2018-05-03 CRAN (R 3.5.0)
#>  ggplot2      * 3.0.0   2018-07-03 CRAN (R 3.5.1)
#>  ggpmisc      * 0.3.0   2018-07-16 CRAN (R 3.5.1)
#>  ggrepel      * 0.8.0   2018-05-09 CRAN (R 3.5.1)
#>  glue         * 1.3.0   2018-07-17 CRAN (R 3.5.1)
#>  graphics     * 3.5.1   2018-07-02 local         
#>  grDevices    * 3.5.1   2018-07-02 local         
#>  grid           3.5.1   2018-07-02 local         
#>  gridExtra      2.3     2017-09-09 CRAN (R 3.5.1)
#>  gtable         0.2.0   2016-02-26 CRAN (R 3.5.1)
#>  haven          1.1.2   2018-06-27 CRAN (R 3.5.1)
#>  here           0.1     2017-05-28 CRAN (R 3.5.1)
#>  Hmisc        * 4.1-1   2018-01-03 CRAN (R 3.5.1)
#>  hms            0.4.2   2018-03-10 CRAN (R 3.5.1)
#>  htmlTable      1.12    2018-05-26 CRAN (R 3.5.1)
#>  htmltools      0.3.6   2017-04-28 CRAN (R 3.5.1)
#>  htmlwidgets    1.2     2018-04-19 CRAN (R 3.5.1)
#>  httr           1.3.1   2017-08-20 CRAN (R 3.5.1)
#>  janeaustenr    0.1.5   2017-06-10 CRAN (R 3.5.1)
#>  jsonlite       1.5     2017-06-01 CRAN (R 3.5.1)
#>  knitr          1.20    2018-02-20 CRAN (R 3.5.1)
#>  labeling       0.3     2014-08-23 CRAN (R 3.5.0)
#>  lattice      * 0.20-35 2017-03-25 CRAN (R 3.5.1)
#>  latticeExtra   0.6-28  2016-02-09 CRAN (R 3.5.1)
#>  lazyeval       0.2.1   2017-10-29 CRAN (R 3.5.1)
#>  lubridate      1.7.4   2018-04-11 CRAN (R 3.5.1)
#>  magrittr       1.5     2014-11-22 CRAN (R 3.5.1)
#>  Matrix         1.2-14  2018-04-13 CRAN (R 3.5.1)
#>  memoise        1.1.0   2017-04-21 CRAN (R 3.5.1)
#>  methods      * 3.5.1   2018-07-02 local         
#>  modelr         0.1.2   2018-05-11 CRAN (R 3.5.1)
#>  munsell        0.5.0   2018-06-12 CRAN (R 3.5.1)
#>  nlme           3.1-137 2018-04-07 CRAN (R 3.5.1)
#>  nnet           7.3-12  2016-02-02 CRAN (R 3.5.1)
#>  pillar         1.3.0   2018-07-14 CRAN (R 3.5.1)
#>  pkgconfig      2.0.2   2018-08-16 CRAN (R 3.5.1)
#>  plyr           1.8.4   2016-06-08 CRAN (R 3.5.1)
#>  polynom        1.3-9   2016-12-08 CRAN (R 3.5.0)
#>  purrr        * 0.2.5   2018-05-29 CRAN (R 3.5.1)
#>  R6             2.2.2   2017-06-17 CRAN (R 3.5.1)
#>  RColorBrewer   1.1-2   2014-12-07 CRAN (R 3.5.0)
#>  Rcpp           0.12.18 2018-07-23 CRAN (R 3.5.1)
#>  readr        * 1.1.1   2017-05-16 CRAN (R 3.5.1)
#>  readxl         1.1.0   2018-04-20 CRAN (R 3.5.1)
#>  rlang          0.2.2   2018-08-16 CRAN (R 3.5.1)
#>  rmarkdown      1.10    2018-06-11 CRAN (R 3.5.1)
#>  rpart          4.1-13  2018-02-23 CRAN (R 3.5.1)
#>  rprojroot      1.3-2   2018-01-03 CRAN (R 3.5.1)
#>  rstudioapi     0.7     2017-09-07 CRAN (R 3.5.1)
#>  rvest          0.3.2   2016-06-17 CRAN (R 3.5.1)
#>  scales       * 1.0.0   2018-08-09 CRAN (R 3.5.1)
#>  SnowballC      0.5.1   2014-08-09 CRAN (R 3.5.0)
#>  splines        3.5.1   2018-07-02 local         
#>  stats        * 3.5.1   2018-07-02 local         
#>  stringi        1.1.7   2018-03-12 CRAN (R 3.5.0)
#>  stringr      * 1.3.1   2018-05-10 CRAN (R 3.5.1)
#>  survival     * 2.42-3  2018-04-16 CRAN (R 3.5.1)
#>  tibble       * 1.4.2   2018-01-22 CRAN (R 3.5.1)
#>  tidyr        * 0.8.1   2018-05-18 CRAN (R 3.5.1)
#>  tidyselect     0.2.4   2018-02-26 CRAN (R 3.5.1)
#>  tidytext     * 0.1.9   2018-05-29 CRAN (R 3.5.1)
#>  tidyverse    * 1.2.1   2017-11-14 CRAN (R 3.5.1)
#>  tokenizers     0.2.1   2018-03-29 CRAN (R 3.5.1)
#>  tools          3.5.1   2018-07-02 local         
#>  utf8           1.1.4   2018-05-24 CRAN (R 3.5.1)
#>  utils        * 3.5.1   2018-07-02 local         
#>  withr          2.1.2   2018-03-15 CRAN (R 3.5.1)
#>  xfun           0.3     2018-07-06 CRAN (R 3.5.1)
#>  xml2           1.2.0   2018-01-24 CRAN (R 3.5.1)
#>  yaml           2.2.0   2018-07-25 CRAN (R 3.5.1)

The current Git commit details are:

#> Local:    master E:/R/caa2018-Tubingen
#> Remote:   master @ origin (https://github.com/benmarwick/caa2018-Tubingen)
#> Head:     [85a0a42] 2018-09-05: oold stuff
=======

This report was generated on 2018-09-10 06:34:35 using the following computational environment and dependencies:

#>  setting  value                       
#>  version  R version 3.5.0 (2018-04-23)
#>  system   x86_64, darwin15.6.0        
#>  ui       X11                         
#>  language (EN)                        
#>  collate  en_US.UTF-8                 
#>  tz       Asia/Ho_Chi_Minh            
#>  date     2018-09-10                  
#> 
#>  package      * version date       source                            
#>  acepack        1.4.1   2016-10-29 CRAN (R 3.5.0)                    
#>  assertthat     0.2.0   2017-04-11 CRAN (R 3.5.0)                    
#>  backports      1.1.2   2017-12-13 CRAN (R 3.5.0)                    
#>  base         * 3.5.0   2018-04-24 local                             
#>  base64enc      0.1-3   2015-07-28 CRAN (R 3.5.0)                    
#>  bindr          0.1.1   2018-03-13 CRAN (R 3.5.0)                    
#>  bindrcpp     * 0.2.2   2018-03-29 CRAN (R 3.5.0)                    
#>  bookdown       0.7     2018-02-18 CRAN (R 3.5.0)                    
#>  broom          0.5.0   2018-07-17 CRAN (R 3.5.0)                    
#>  cellranger     1.1.0   2016-07-27 CRAN (R 3.5.0)                    
#>  checkmate      1.8.5   2017-10-24 CRAN (R 3.5.0)                    
#>  cli            1.0.0   2017-11-05 CRAN (R 3.5.0)                    
#>  cluster        2.0.7-1 2018-04-13 CRAN (R 3.5.0)                    
#>  colorspace     1.3-2   2016-12-14 CRAN (R 3.5.0)                    
#>  compiler       3.5.0   2018-04-24 local                             
#>  crayon         1.3.4   2017-09-16 CRAN (R 3.5.0)                    
#>  data.table     1.11.4  2018-05-27 CRAN (R 3.5.0)                    
#>  datasets     * 3.5.0   2018-04-24 local                             
#>  devtools       1.13.6  2018-06-27 cran (@1.13.6)                    
#>  digest         0.6.15  2018-01-28 CRAN (R 3.5.0)                    
#>  dplyr        * 0.7.6   2018-06-29 cran (@0.7.6)                     
#>  evaluate       0.11    2018-07-17 cran (@0.11)                      
#>  fansi          0.2.3   2018-05-06 CRAN (R 3.5.0)                    
#>  forcats      * 0.3.0   2018-02-19 CRAN (R 3.5.0)                    
#>  foreign        0.8-70  2017-11-28 CRAN (R 3.5.0)                    
#>  Formula      * 1.2-3   2018-05-03 CRAN (R 3.5.0)                    
#>  ggplot2      * 3.0.0   2018-07-03 cran (@3.0.0)                     
#>  ggpmisc      * 0.3.0   2018-07-16 CRAN (R 3.5.0)                    
#>  ggrepel      * 0.8.0   2018-05-09 CRAN (R 3.5.0)                    
#>  glue         * 1.3.0   2018-08-09 Github (tidyverse/glue@a292148)   
#>  graphics     * 3.5.0   2018-04-24 local                             
#>  grDevices    * 3.5.0   2018-04-24 local                             
#>  grid           3.5.0   2018-04-24 local                             
#>  gridExtra      2.3     2017-09-09 CRAN (R 3.5.0)                    
#>  gtable         0.2.0   2016-02-26 CRAN (R 3.5.0)                    
#>  haven          1.1.1   2018-01-18 CRAN (R 3.5.0)                    
#>  here           0.1     2017-05-28 CRAN (R 3.5.0)                    
#>  highr          0.6     2016-05-09 CRAN (R 3.5.0)                    
#>  Hmisc        * 4.1-1   2018-01-03 CRAN (R 3.5.0)                    
#>  hms            0.4.2   2018-03-10 CRAN (R 3.5.0)                    
#>  htmldeps       0.1.1   2018-09-03 Github (rstudio/htmldeps@c1023e0) 
#>  htmlTable      1.12    2018-05-26 CRAN (R 3.5.0)                    
#>  htmltools      0.3.6   2017-04-28 CRAN (R 3.5.0)                    
#>  htmlwidgets    1.2     2018-04-19 CRAN (R 3.5.0)                    
#>  httr           1.3.1   2017-08-20 CRAN (R 3.5.0)                    
#>  janeaustenr    0.1.5   2017-06-10 CRAN (R 3.5.0)                    
#>  jsonlite       1.5     2017-06-01 CRAN (R 3.5.0)                    
#>  knitr          1.20    2018-02-20 CRAN (R 3.5.0)                    
#>  labeling       0.3     2014-08-23 CRAN (R 3.5.0)                    
#>  lattice      * 0.20-35 2017-03-25 CRAN (R 3.5.0)                    
#>  latticeExtra   0.6-28  2016-02-09 CRAN (R 3.5.0)                    
#>  lazyeval       0.2.1   2017-10-29 CRAN (R 3.5.0)                    
#>  lubridate      1.7.4   2018-04-11 CRAN (R 3.5.0)                    
#>  magrittr       1.5     2014-11-22 CRAN (R 3.5.0)                    
#>  Matrix         1.2-14  2018-04-13 CRAN (R 3.5.0)                    
#>  memoise        1.1.0   2017-04-21 CRAN (R 3.5.0)                    
#>  methods      * 3.5.0   2018-04-24 local                             
#>  modelr         0.1.2   2018-05-11 CRAN (R 3.5.0)                    
#>  munsell        0.5.0   2018-06-12 cran (@0.5.0)                     
#>  nlme           3.1-137 2018-04-07 CRAN (R 3.5.0)                    
#>  nnet           7.3-12  2016-02-02 CRAN (R 3.5.0)                    
#>  pillar         1.3.0   2018-07-14 CRAN (R 3.5.0)                    
#>  pkgconfig      2.0.1   2017-03-21 CRAN (R 3.5.0)                    
#>  plyr           1.8.4   2016-06-08 CRAN (R 3.5.0)                    
#>  png            0.1-7   2013-12-03 CRAN (R 3.5.0)                    
#>  polynom        1.3-9   2016-12-08 CRAN (R 3.5.0)                    
#>  purrr        * 0.2.5   2018-05-29 CRAN (R 3.5.0)                    
#>  R6             2.2.2   2017-06-17 CRAN (R 3.5.0)                    
#>  RColorBrewer   1.1-2   2014-12-07 CRAN (R 3.5.0)                    
#>  Rcpp           0.12.18 2018-07-23 cran (@0.12.18)                   
#>  readr        * 1.1.1   2017-05-16 CRAN (R 3.5.0)                    
#>  readxl         1.1.0   2018-04-20 CRAN (R 3.5.0)                    
#>  rlang          0.2.1   2018-05-30 CRAN (R 3.5.0)                    
#>  rmarkdown      1.10.12 2018-09-03 Github (rstudio/rmarkdown@6e56ad8)
#>  rpart          4.1-13  2018-02-23 CRAN (R 3.5.0)                    
#>  rprojroot      1.3-2   2018-01-03 CRAN (R 3.5.0)                    
#>  rstudioapi     0.7     2017-09-07 CRAN (R 3.5.0)                    
#>  rvest          0.3.2   2016-06-17 CRAN (R 3.5.0)                    
#>  scales       * 1.0.0   2018-08-09 cran (@1.0.0)                     
#>  SnowballC      0.5.1   2014-08-09 CRAN (R 3.5.0)                    
#>  splines        3.5.0   2018-04-24 local                             
#>  stats        * 3.5.0   2018-04-24 local                             
#>  stringi        1.2.4   2018-07-20 cran (@1.2.4)                     
#>  stringr      * 1.3.1   2018-05-10 CRAN (R 3.5.0)                    
#>  survival     * 2.42-3  2018-04-16 CRAN (R 3.5.0)                    
#>  tibble       * 1.4.2   2018-01-22 CRAN (R 3.5.0)                    
#>  tidyr        * 0.8.1   2018-05-18 CRAN (R 3.5.0)                    
#>  tidyselect     0.2.4   2018-02-26 CRAN (R 3.5.0)                    
#>  tidytext     * 0.1.9   2018-05-29 CRAN (R 3.5.0)                    
#>  tidyverse    * 1.2.1   2017-11-14 CRAN (R 3.5.0)                    
#>  tokenizers     0.2.1   2018-03-29 CRAN (R 3.5.0)                    
#>  tools          3.5.0   2018-04-24 local                             
#>  utf8           1.1.4   2018-05-24 CRAN (R 3.5.0)                    
#>  utils        * 3.5.0   2018-04-24 local                             
#>  withr          2.1.2   2018-03-15 CRAN (R 3.5.0)                    
#>  xfun           0.3     2018-07-06 CRAN (R 3.5.0)                    
#>  xml2           1.2.0   2018-01-24 CRAN (R 3.5.0)                    
#>  yaml           2.2.0   2018-07-25 cran (@2.2.0)

The current Git commit details are:

#> Local:    master /Users/bmarwick/Desktop/caa2018-Tubingen
#> Remote:   master @ origin (https://github.com/benmarwick/caa2018-Tubingen)
#> Head:     [c52f218] 2018-09-10: Merge pull request #5 from SCSchmidt/discussion
>>>>>>> a4f724d0933cfc4b2322ddb3b951beb784e55322